Recently, for broadcasting stations and homes, such devices have been spread that adopt an encoding system such as MPEG (Moving Picture Experts Group) to encode (compress) successive images through orthogonal transformation such as discrete cosine transform, and motion compensation, utilizing the redundancy of the successive image information, for efficient information transmission or storage, by taking the successive images as digital information.
Especially, the MPEG2 (ISO/IEC 13818-2) encoding system is defined as a general image encoding system, and is widely used as an application for professionals and for consumers since it can treat interlaced images and progressively scanned images, and standard resolution images and high resolution images. By using this MPEG2 encoding system, high encoding efficiency (compression rate) and high quality of images can be provided, for example, by assigning interlaced images of standard resolution of 720×480 pixels an amount of encoding (bit rate) of 4 to 8 [Mbps], or by assigning progressively scanned images of high resolution of 1920×1088 pixels a bit rate of 18 to 22 [Mbps].
The MPEG2 encoding system is mainly used for encoding high quality of images for broadcasting and does not cope with an amount of encoding (bit rate) lower than that used by the MPEG1 encoding system, that is, an encoding method with high encoding efficiency. It was expected that popularization of mobile terminals brings high needs of such an encoding system, and therefore the MPEG4 encoding system was standardized. The MPEG4 encoding system for images was approved as international standard ISO/IEC 14496-2 in December 1998.
In addition, recently, an encoding system (hereinafter referred to as JTV encoding system) called MPEG4AVC or H.264 was standardized by a joint video team composed of a VCEG group and an MPEG group. Compared with the MPEG2 and the MPEG4, this JVT encoding system can provide higher encoding efficiency although it requires more operations for encoding and decoding.
Now, FIG. 8 shows a rough construction of an encoding apparatus which realizes an encoding process with any of the encoding systems referred above. As shown in FIG. 8, the encoding apparatus 100 is composed of an image rearrangement buffer 102, an adder 103, an orthogonal transformation unit 104, a quantization unit 105, a reverse encoding unit 106, a storage buffer 107, a dequantization unit 108, an inverse orthogonal transformation unit 109, a frame memory 110, a motion prediction/compensation unit 111 and a rate control unit 112.
In this case, the encoding apparatus 100 stores successive image information in the image rearrangement buffer 102 to rearrange the successive image information according to GOP (Group of Pictures) structure on a unit-image-information basis (frame by frame or field by field).
The image rearrangement buffer 102 gives the orthogonal transformation unit 104 unit image information out of the successive image information, which should be intra-prediction-encoded. The orthogonal transformation unit 104 applies orthogonal transformation such as the discrete cosine transform or the Karhunen Loeve transform, to the unit image information and gives an obtained orthogonal transformation coefficient to the quantization unit 105.
The quantization unit 105 performs a quantization process on the orthogonal transformation coefficient given from the orthogonal transformation unit 104, under the control of the rate control unit 112, and supplies obtained quantized information (a quantized orthogonal transformation coefficient) to the reverse encoding unit 106 and the dequantization unit 108. The reverse encoding unit 106 applies variable-length coding or reverse encoding such as arithmetic coding to the quantized information, and stores obtained encoded information (encoded quantized-information) in the storage buffer 107.
The dequantization unit 108 applies a dequantization process to the quantized information and supplies obtained orthogonal transformation coefficient to the inverse orthogonal transformation unit 109. The inverse orthogonal transformation unit 109 applies the inverse orthogonal transformation to the orthogonal transformation coefficient and stores, if necessary, obtained unit image information in the frame memory 110 as reference image information.
On the other hand, the image rearrangement buffer 102 supplies unit image information which should be inter-prediction-encoded, out of the successive image information to the motion prediction/compensation unit 111. The motion prediction/compensation unit 111 performs a motion prediction/compensation process by using the unit image information and reference image information read from the frame memory 10, and supplies obtained predicted image information to the adder 103. The adder 103 supplies to the orthogonal transformation unit 104 difference between the predicted image information and corresponding unit image information as differential information.
This differential information is subjected to various processes, as in the case of the intra-encoding, and the resultant is stored in the storage buffer 107 as encoded information and is stored, if necessary, in the frame memory 110 as reference image information.
In addition, the motion compensation/prediction unit 111 gives the reverse encoding unit 106 motion vector information which is obtained together with the predicted image information as a result of the motion prediction/compensation process. The reverse encoding unit 106 performs the reverse encoding process on the motion vector information to thereby obtain encoded motion vector information for the header part of the corresponding encoded information.
In such a manner, the encoding apparatus 100 successively creates encoded information on a unit-image-information basis by performing the encoding process on the successive image information, and successively outputs the encoded information via the storage buffer 107.
Next, FIG. 9 shows a rough construction of a decoding apparatus which performs a decoding process corresponding to the encoding system of the encoding apparatus 100. As shown in FIG. 9, the decoding apparatus 120 is composed of a storage buffer 121, a reverse decoding unit 122, a dequantization unit 123, an inverse orthogonal transformation unit 124, an adder 125, an image rearrangement buffer 126, a motion prediction/compensation unit 127, and a frame memory 128.
In this case, the decoding apparatus 120 temporarily stores encoded information which is successively inputted, in the storage buffer 121 and then supplies it to the reverse decoding unit 122. In a case where the encoded information have been subjected to the intra-prediction encoding, the reverse decoding unit 122 applies a decoding process, variable-length decoding or arithmetic decoding, to the encoded information, and supplies obtained quantized information to the dequantization unit 123.
The dequantization unit 123 applies a dequantization process to the quantized information given from the reverse decoding unit 122 and supplies obtained orthogonal transformation coefficient to the inverse orthogonal transformation unit 124. The inverse orthogonal transformation unit 124 applies an inverse orthogonal transformation process to the orthogonal transformation coefficient to thereby create the original image information before the encoding process (hereinafter, referred to as restored image information), and stores this in the image rearrangement buffer 126.
On the other hand, in a case where the encoded information have been subjected to the inter prediction encoding, the reverse decoding unit 122 performs a decoding process on both of this encoded information and the encoded motion vector information which has been inserted in the header part of the encoded information, and supplies obtained quantized information to the dequantization unit 123 and supplies the motion vector information to the motion prediction/compensation unit 127. The quantized information is subjected to various processes, as in the case of decoding encoded information intra-encoded, and is then supplied to the adder 125 as differential information.
In addition, the motion prediction/compensation unit 127 creates predicted image information based on the motion vector information and reference image information stored in the frame memory 128, and supplies this to the adder 125. The adder 125 synthesizes the reference image information and the differential information, and stores obtained restored image information to the image rearrangement buffer 126.
In the aforementioned manner, the decoding apparatus 120 successively creates restored image information by performing the decoding process on the encoded information successively inputted, and successively outputs the restored image information via the image rearrangement buffer 126 to, for example, a display unit (not shows) for successive reproduction.
By the way, since the MPEG2 encoding system prescribes that only I (Intra)-pictures and P (Predictive)-pictures are used as pictures for inter-prediction-encoding, a decoding order for the decoding process is naturally determined.
Therefore, in a case where the decoding apparatus 120 successively reproduces restored image information which is created by performing the decoding process on encoded information with the MPEG 2 encoding system, it can appropriately display images based on the restored image information on the display unit without adjusting the output timing of the restored image information at the image rearrangement buffer 126.
On the other hand, as compared with the MPEG2 encoding system, the JVT encoding system has a larger degree of freedom for selection of pictures for prediction-encoding, for example, it can treat not only I- and P-pictures but also B (Bidirectional)-pictures as pictures for inter-prediction-encoding.
However, the JVT encoding system does not prescribe a decoding order in a decoding process and further does not specify output timing of restored image information.
Therefore, if the decoding apparatus 120 successively reproduces restored image information which is created by performing the decoding process on encoded information with the JVT encoding system, such happening occurs that encoded information is still being decoded at the output timing for restored image information corresponding to the encoded information due to a limited resource of the image rearrangement buffer 126, and as a result, the continuousness is broken.